A Quality Control Framework for Gold Standard Reference Translations: The Process and Toolkit Developed for GALE
نویسندگان
چکیده
While a multitude of machine translation (MT) evaluation metrics exist, most require one or more gold standard references. For the DARPA GALE program, source data is translated according to detailed guidelines and high quality standards, but these raw translations then undergo a rigorous and carefully constructed quality control (QC) process to create the final references. GALE evaluation translation references undergo several distinct phases of translation and quality control, utilizing a specially-designed toolkit, QCTool. This paper describes this framework and suggests that it may be successfully applied to improve and streamline other large-scale translation projects.
منابع مشابه
Contrastive analysis of diagnostic tests evaluation without gold stand-ard: review article
Considering the advancement of medical sciences, diagnostic tests have been developed to distinguish patients from healthy population. Therefore, Determining and evaluation of the diagnostic accuracy tests is of great importance. The accuracy of a test under evaluation is determined through the amount of agreement between its results with the results of the gold standard, and this test accuracy...
متن کاملProcess Capability Studies in an Automated Flexible Assembly Process: A Case Study in an Automotive Industry
Statistical Process Control (SPC) methods can significantly increase organizational efficiency if appropriately used. The primary goal of process capability studies is to obtain critical information about processes to render them even more effective. This paper proposes a comprehensive framework for proper implementation of SPC studies, including the design of the sampling procedure and interva...
متن کاملDeveloping customer relationship management (CRM) processes using standard frameworks and MCDM techniques
Organizations have found that functional approach to business destroys flexibility and agility by gaining experience over the time. The main weakness with Task-based organizations is that they can hardly act flexibly and adapt themselves to the changing environment. In today’s' dynamic and competitive business world, organizations focus significantly on managing and improving their business pro...
متن کاملGiving the Power to Bilingual Speakers Position paper for the APE Workshop
Post-editing has traditionally been used to improve the quality of MT output to make it usable. More recently, post-editing has also been shown to be useful for automatic evaluation of MT systems. As an alternative to current automatic evaluation metrics, such as BLEU and NIST, which are criticized for not having a high correlation with human judgments, minimal post-editing of MT output provide...
متن کاملConstruction of α-cut fuzzy X control charts based on standard deviation and range using fuzzy triangular numbers
Control charts are one of the most important tools in statistical process control that lead to improve quality processes and ensure required quality levels. In traditional control charts, all data should be exactly known, whereas there are many quality characteristics that cannot be expressed in numerical scale, such as characteristics for appearance, softness, and color. Fuzzy sets theory is a...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2011